In recent years, an attention has been drawn on reconstruction of a 3D moving scene. A great success has been achieved on, for example, a gaming product that serves as a device-free interface by measuring a human body in real time, and analyzing the motion of the human body (see, for example, NPL 1). Further, a research for employing such a product as the eyes of an autonomous mobile robot has been continued, and the importance of measurement of a moving object has been strongly noticed. As for currently employed moving object scanners, 3D scanners that measure static scenes cannot perform shape measurement as accurately and densely as existing scanners. However, if improvement of the accuracy and resolution is realized, these scanners should be more useful for various purposes, such as medical application and fluid analysis.
There are multiple methods present for measuring the shapes of moving objects, such as stereo methods using only cameras and laser scanning methods using Time-of-Flight (TOF) systems. Especially, a method for emitting structured light using a system that employs a projector and a camera is suitable for obtaining shape data of a moving object, and development and research for this method has been popular (see, for example, NPL1 to NPL4).
Structured-light projection methods are usually classified into two types: temporal-encoding methods and spatial-encoding methods. Since a spatial-encoding method is a method for performing shape reconstruction (one-shot scanning) based on a single image, it is ideal to measure a moving object at a high frame rate. Therefore, many researches have been involved in spatial-encoding methods. According to the spatial-encoding method, correspondence information that can be uniquely specified among the entire projected pattern is embedded directly in a two-dimensional pattern. An appropriately large area is required for this process, and therefore, the resolution for reconstruction tends to be low. Furthermore, decoding errors tend to occur due to, for example, distortion of a pattern caused by the change of the surface shape.
One of the methods available for efficiently embedding correspondence information in a two-dimensional pattern is the use of a color code. A method for employing multiple colors to embed a plurality of sets of bit data in individual points has been widely used (see, for example, NPL 3 and 5 to 8). However, in a case wherein color information is employed, it is required that the individual RGB color components be appropriately reflected on the surface of a target object. Further, for projectors available on the market, spectral distributions of the individual color components are overlapped each other, and therefore, an error tends to occur in determination of colors for individual pixels. To avoid this problem, a method using dot patterns or grid patterns have been proposed as a spatial-encoding method that does not use colors. However, the problems on ambiguities of correspondences and sparse reconstruction have not yet been resolved.
Generally, systems employing TOF scanners or active stereos are popular as active measurement systems. Further, various methods for active measurement of a moving object have been researched. In many TOF laser scanners, a point laser beam is projected to an object to be measured, and the interval time required until the laser beam returns to a detector is measured. Since measurement is performed for one point at a time, it is unsuitable for measurement of a large region in a short period of time. To measure a moving object, etc., there are devices proposed that project temporally-modulated light to a large area, observe the modulation of the light for the individual pixels of a 2D sensor, and acquire a depth image (see, for example, NPL 9 and 10). However, the present systems are easily affected by the interference of other light sources, and the resolution is lower than that for the normal cameras.
As for the measurement using the active stereo, in many cases, point laser beams or line laser beams are projected to an object, which is then scanned for measurement. This method is unsuitable for measurement of a moving object, because an extended period is required for measurement. The measurement period can be reduced by employing a planar light source, such as a video projector; however, a problem on ambiguity on correspondences must be resolved. For resolving the problem, there are typically two solutions, i.e., a temporal-encoding method and a spatial encoding method (see, for example, NPL 5).
According to the temporal-encoding method, multiple patterns are projected, and information is encoded in the temporal modulations of the individual points of the pattern. Thus, it is essentially unsuitable for measuring a moving object. To compensate for the shortcomings, there have been some methods proposed. For example, a method for changing the pattern with high frequencies (see, for example, NPL 11), a method for reducing the required number of patterns by using phase patterns (see, for example, NPL 12) and a method employing DMD patterns (see, for example, NPL 13) have been proposed.
As an approach slightly different from the normal active stereo, a spacetime stereo method, for example, has been proposed, whereby two or more cameras are employed to project a pattern that temporally changes (see, for example, NPL 14). At present, an example wherein measurement around 100 fps was successfully performed by employing motion estimation has also been introduced. However, since information for multiple frames is required, the method is not appropriate for measurement of an object that moves fast.
The spatial-encoding method is appropriate for measurement of a moving object, because the shape of an object is reconstructed by using a static pattern and based on only a single input image. However, since information must be embedded in certain spatial areas of the pattern, the resolution tends to be low. Moreover, determination of correspondences tends to be unstable because the patterns are distorted due to the color and the shape of the object surface. Therefore, many methods have been proposed to solve the problems. For example, a method using multiple color bands to avoid the same combinations of colors (see, for example, NPL 15 and 16), a method for employing unique dotted lines (see, for example, NPL 17 and 18) and a method for embedding information in a two-dimensional pattern (see, for example, NPL 1 and 19). However, there is not yet a method proposed whereby sufficient performances are provided in all aspects of precision, resolution, and stability.